Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](cloud) Fix cloud decomission and check wal #47187

Merged
merged 3 commits into from
Jan 29, 2025
Merged

[fix](cloud) Fix cloud decomission and check wal #47187

merged 3 commits into from
Jan 29, 2025

Conversation

deardeng
Copy link
Contributor

@deardeng deardeng commented Jan 17, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@deardeng deardeng marked this pull request as draft January 17, 2025 12:51
@deardeng
Copy link
Contributor Author

wait regression

@deardeng deardeng marked this pull request as ready for review January 21, 2025 09:25
@deardeng
Copy link
Contributor Author

run builldall

@deardeng
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32199 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 733532e56fb1ce180f90d3526cd7cfbea6f5838c, data reload: false

------ Round 1 ----------------------------------
q1	17578	5463	5394	5394
q2	2036	310	169	169
q3	10432	1200	723	723
q4	10267	970	509	509
q5	8640	2385	2140	2140
q6	190	168	137	137
q7	905	747	616	616
q8	9233	1322	1064	1064
q9	5108	4841	4877	4841
q10	6828	2361	1984	1984
q11	461	304	269	269
q12	355	361	235	235
q13	17764	3689	3115	3115
q14	229	232	209	209
q15	517	479	467	467
q16	639	617	581	581
q17	566	853	316	316
q18	6929	6475	6506	6475
q19	2382	947	530	530
q20	296	322	194	194
q21	2741	2136	1924	1924
q22	359	335	307	307
Total cold run time: 104455 ms
Total hot run time: 32199 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5627	5531	5538	5531
q2	234	336	237	237
q3	2217	2645	2293	2293
q4	1410	1799	1371	1371
q5	4308	4716	4649	4649
q6	165	165	133	133
q7	2026	1938	1826	1826
q8	2588	2762	2729	2729
q9	7243	7181	7252	7181
q10	3023	3293	2813	2813
q11	571	516	476	476
q12	663	727	611	611
q13	3533	3914	3274	3274
q14	288	298	291	291
q15	516	509	470	470
q16	641	698	641	641
q17	1225	1730	1239	1239
q18	7552	7502	7285	7285
q19	794	1145	1056	1056
q20	1993	2017	1925	1925
q21	5775	5261	4921	4921
q22	600	586	557	557
Total cold run time: 52992 ms
Total hot run time: 51509 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 194553 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 733532e56fb1ce180f90d3526cd7cfbea6f5838c, data reload: false

query1	1302	947	933	933
query2	6318	2084	2037	2037
query3	11082	4647	4570	4570
query4	32412	23308	22953	22953
query5	3676	593	461	461
query6	273	193	211	193
query7	3985	482	325	325
query8	283	231	217	217
query9	9497	2577	2589	2577
query10	450	310	252	252
query11	17751	15267	15096	15096
query12	164	101	103	101
query13	1557	516	414	414
query14	9826	7557	7340	7340
query15	264	219	187	187
query16	8488	663	509	509
query17	1642	772	603	603
query18	2148	417	352	352
query19	214	197	166	166
query20	124	125	113	113
query21	207	135	107	107
query22	4481	4587	4247	4247
query23	34975	33252	33469	33252
query24	6460	2341	2292	2292
query25	457	459	407	407
query26	713	254	153	153
query27	2258	482	342	342
query28	5596	2464	2455	2455
query29	542	557	434	434
query30	226	201	151	151
query31	941	914	813	813
query32	83	57	59	57
query33	480	352	301	301
query34	779	895	527	527
query35	837	847	785	785
query36	1034	1052	966	966
query37	120	99	76	76
query38	4385	4444	4278	4278
query39	1511	1459	1442	1442
query40	202	116	106	106
query41	55	48	51	48
query42	117	105	101	101
query43	529	547	518	518
query44	1300	816	830	816
query45	185	186	182	182
query46	891	1061	670	670
query47	1931	1913	1854	1854
query48	400	409	318	318
query49	735	495	419	419
query50	656	668	394	394
query51	7102	7050	7010	7010
query52	105	111	95	95
query53	233	258	187	187
query54	477	510	435	435
query55	104	84	84	84
query56	272	257	250	250
query57	1217	1201	1154	1154
query58	271	232	242	232
query59	3126	3235	3102	3102
query60	290	270	256	256
query61	119	119	116	116
query62	784	695	650	650
query63	232	191	185	185
query64	3187	1008	663	663
query65	3245	3146	3203	3146
query66	820	401	299	299
query67	15776	15652	15427	15427
query68	7354	825	544	544
query69	489	291	274	274
query70	1247	1160	1155	1155
query71	381	287	256	256
query72	6215	3997	3819	3819
query73	648	740	434	434
query74	9964	9163	8957	8957
query75	3177	3146	2662	2662
query76	3149	1182	765	765
query77	527	377	280	280
query78	10382	9976	9264	9264
query79	3494	803	584	584
query80	694	524	445	445
query81	517	270	230	230
query82	652	151	124	124
query83	161	166	148	148
query84	247	87	74	74
query85	760	351	308	308
query86	392	314	296	296
query87	4408	4473	4336	4336
query88	4744	2191	2137	2137
query89	392	322	298	298
query90	1710	191	190	190
query91	133	137	108	108
query92	61	56	53	53
query93	1722	830	539	539
query94	684	401	298	298
query95	324	269	261	261
query96	494	616	282	282
query97	2840	2862	2752	2752
query98	230	194	192	192
query99	1436	1339	1287	1287
Total cold run time: 291530 ms
Total hot run time: 194553 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 733532e56fb1ce180f90d3526cd7cfbea6f5838c, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.04
query3	0.24	0.08	0.07
query4	1.62	0.11	0.11
query5	0.42	0.43	0.42
query6	1.17	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.02
query9	0.61	0.51	0.51
query10	0.55	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.60	0.61	0.59
query14	2.75	2.86	2.83
query15	0.91	0.82	0.84
query16	0.40	0.38	0.39
query17	1.05	1.09	1.06
query18	0.22	0.21	0.21
query19	1.96	1.86	2.00
query20	0.02	0.00	0.01
query21	15.38	0.89	0.58
query22	0.75	0.76	0.60
query23	15.40	1.42	0.56
query24	2.86	1.73	1.97
query25	0.24	0.20	0.09
query26	0.19	0.13	0.14
query27	0.07	0.04	0.04
query28	14.67	0.97	0.43
query29	12.60	3.97	3.34
query30	0.25	0.08	0.06
query31	2.86	0.57	0.38
query32	3.24	0.55	0.46
query33	3.07	2.98	3.02
query34	16.68	5.19	4.56
query35	4.54	4.58	4.59
query36	0.68	0.49	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.03	0.03
query40	0.17	0.13	0.12
query41	0.07	0.03	0.03
query42	0.04	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.92 s
Total hot run time: 31.7 s

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 23, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@gavinchou gavinchou merged commit 60910a4 into apache:master Jan 29, 2025
26 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 29, 2025
@deardeng
[fix](cloud) Fix cloud decomission and check wal (#47187)
2c8e506
@github-actions github-actions bot mentioned this pull request Jan 29, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jan 30, 2025
@github-actions @deardeng
branch-3.0: [fix](cloud) Fix cloud decomission and check wal #47187 (#…
4799310 
…47483)

Cherry-picked from #47187

Co-authored-by: deardeng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gavinchou gavinchou gavinchou approved these changes

@mymeiyi mymeiyi mymeiyi approved these changes

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

@w41ter w41ter Awaiting requested review from w41ter w41ter is a code owner

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.4-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@deardeng @hello-stephen @doris-robot @mymeiyi @gavinchou @dataroaring